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ABSTRACT 

In 1995 the School Board of Philadelphia adopted Children 
Achieving, a systemic reform initiative. This report is an evaluation of that 
reform. The report examines the role of the central office and describes its 
evolution over the course of the program. The first section recounts how 
conflicts arose over theories of systemic reform, underlying beliefs and 
values of reformers, and the new role of the central office in the reform 
plan. The second section examines the capacity of the school district to 
effectively support the reform and discusses the contextual issues that 
affected implementation. The evaluation used data on student test scores, 
promotion and graduation rates, and student and teacher attendance; school 
indicators describing teacher and student characteristics; and surveys and 
interviews of teachers, students, administrators, central-office staff, 
parents, and community members. The report concludes that Children Achieving 
was neither an unbridled success nor an irredeemable failure. Test scores 
improved, standards and curriculum frameworks were created for every grade, 
and accountability measures were designed. At the same time, flaws in the 
implementation, lack of capacity building, and inconsistency by stakeholders 
over the values underlying the reform led to the reform* s demise. (Contains 
92 footnotes.) (WFA) 
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ABOUT THE 
CHILDREN 
ACHIEVING 
CHALLENGE 

In February 1995 shortly after the 
School Board of Philadelphia adopted 
Children Achieving as a systemic reform 
agenda to improve the Philadelphia 
public schools, the Annenberg 
Foundation designated Philadelphia as 
one of a few American cities to receive 
a five-year $50 million Annenberg 
Challenge grant to improve public 
education. 

Among the conditions for receiving the 
grant was a requirement to raise two 
matching dollars ($100 million over five 
years) for each one received from the 
Annenberg Foundation and to create 
an independent management structure 
to provide program, fiscal, and 
evaluation oversight of the grant. In 
Philadelphia, a business organization, 
Greater Philadelphia First, assumed this 
responsibility, and with it, the challenge 
of building and sustaining civic support 
for the improvement of public 
education in the city. 

Philadelphia's Children Achieving was a 
sweeping systemic reform initiative. 
Systemic reform eschews a school-by- 
school approach to reform and relies on 
coherent policy, improved coordination 
of resources and services, content and 
performance standards, 
decentralization of decision-making, 
and accountability mechanisms to 
transform entire school systems. Led by 



a dynamic superintendent and central 
office personnel, Children Achieving 
was the first attempt by an urban 
district to test systemic reform in 
practice. 

EVALUATION OF 

CHILDREN 

ACHIEVING 

In 1996 the Consortium for Policy 
Research in Education (CPRE) at the 
University of Pennsylvania and its 
partner, Research for Action (RFA) were 
charged by the Children Achieving 
Challenge with the evaluation of 
Children Achieving. Between the 1 995- 
1996 and 2000-2001 school years, 

CPRE and RFA researchers interviewed 
hundreds of teachers, principals, 
parents, students, district officials, and 
civic leaders; sat in on meetings where 
the plan was designed, debated, and 
revised; observed its implementation in 
classrooms and schools; conducted two 
system-wide surveys of teachers; and 
carried out independent analyses of the 
District's test results and other 
indicators of system performance. An 
outline of the research methods used 
by CPRE and RFA is included in this 
report. A listing of the reports on 
Children Achieving currently available 
from CPRE is found below. There will 
be several additional reports released 
in the coming months. New reports will 
be listed and available as they are 
released on the CPRE web site at 
www.gse.upenn.edu/cpre/. 
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CHILDREN 
ACHIEVING'S 
THEORY OF ACTION 

To assess the progress and effects of a 
comprehensive reform such as Children 
Achieving, it is essential to understand 
its "theory of action," that is, the 
assumptions made about what actions 
or behaviors will produce the desired 
effects. A summary of the Children 
Achieving theory of action follows: 

Given high academic standards and 
strong incentives to focus their efforts 
and resources; more control over 
school resource allocations, 
organization, policies, and programs; 
adequate funding and resources; more 
hands-on leadership and high-quality 
support; better coordination of 
resources and programs; schools 
restructured to support good teaching 
and encourage improvement of 
practice; rich professional development 
of their own choosing; and increased 
public understanding and support; the 
teachers and administrators of the 
Philadelphia schools will develop, 
adopt, or adapt instructional 
technologies and patterns of behavior 
that will help all children reach the 
District's high standards. 

ADDITIONAL 
READING ON 
CHILDREN 
ACHIEVING 



are currently available through CPRE at 
(215) 573-0700. 

• Recruiting and Retaining Teachers: 
Keys to Improving the Philadelphia 
Public Schools (May 2001) 

• School Leadership and Reform: 

Case Studies of Philadelphia 
Principals (May 2001) 

• Contradictions and Control in 
Systemic Reform: The Ascendancy 
of the Central Office in Philadelphia 
Schools (August 2001) 

• Clients, Consumers, or 
Collaborators? Parents and their 
Roles in School Reform During 
Children Achieving, 1 995-2000 
(August 2001) 

DISCLAIMER 

The research reported herein was 
conducted by the Consortium for Policy 
Research in Education and Research for 
Action. Funding for this work was 
provided by Greater Philadelphia First 
and The Pew Charitable Trusts. 
Opinions expressed in this report are 
those of the author, and do not 
necessarily reflect the views of Greater 
Philadelphia First, The Pew Charitable 
Trusts, or the institutional partners of 
CPRE. 
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CHILDREN 
ACHIEVING 
EVALUATION 
1 995 - 2001 : 

RESEARCH METHODS 

During the past five years, the 
Consortium for Policy Research in 
Education and Research for Action used 
the research methods indicated below 
in their evaluation of the Children 
Achieving Challenge. 

1. 1996-2000 school-level data on 
indicators that made up the 
District's Performance Responsibility 
Index including student scores on 
the SAT-9, student promotion and 
graduation rates, student 
attendance, and teacher 
attendance. 

2. Two census surveys of teachers, the 
first in 1997 and the second in 1999. 
Teachers were asked about reform 
implementation, school conditions, 
and teaching practices. There was a 
greater than 60 percent response 
rate on both surveys. 

3. School indicators describing teacher 
and student characteristics in 1996 
and 1999 obtained from the School 
District of Philadelphia's Information 
Services. These data included school 
enrollment, number of teachers, the 
proportion of students qualifying for 
free or reduced price lunch, among 
other indicators. These data were 
used for descriptive purposes and in 
hierarchical linear and logistic 
regression models to help 



understand the relationships among 
reform implementation, student 
outcomes, and school 
characteristics. 

4. Five years (1995-1996 through 1999- 
2000) of qualitative research in 49 
schools (26 elementary, 11 middle, 
and 12 high schools) in 14 clusters. 
Qualitative research included: 
interviews of teachers, principals, 
parents, outside partners who 
worked in the schools, and in a few 
cases, students; observations of 
classrooms, SLC meetings, 
professional development sessions, 
and school leadership team 
meetings; and review of school 
documents (School Improvement 
Plan, budget, etc.). Intensive, multi- 
year case study research in a subset 
of 25 schools (1 3 elementary, 5 
middle, and 7 high schools). 

5. Interviews of central office and 
cluster staff and observations of 
meetings and other events. 

6. Interviews of 40 Philadelphia civic 
leaders (included political leaders, 
leaders in the funding community, 
public education advocates, 
journalists, and business leaders). 

In addition, numerous other studies 
conducted during Children Achieving 
informed this evaluation. These 
included: Bruce Wilson and Dick 
Corbett's three-year interview study of 
middle school students; an evaluation 
of the Philadelphia Urban Systemic 
Initiative in Mathematics and Science 
conducted by Research for Action; the 
Philadelphia Education Longitudinal 
Study conducted by Frank Furstenberg 
at the University of Pennsylvania; and 
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the evaluation of the William Penn 
Foundation's initiative in two clusters 
conducted by the National Center for 
Restructuring Education, Schools, and 
Teaching. 
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INTRODUCTION: 
THE ROLE OF THE 
CENTRAL OFFICE IN 
REFORM 



T he central administrators of 

school districts have only rarely 
been the focus of education 
research. Still, the prevailing 
attitude is that districts are either 
irrelevant to or inhibitors of reform. 
Studies of school-based reform efforts 
often conclude that central office 
policies impede innovation. The 
proposals for school choice, charter 
schools, vouchers, and contract schools 
are a response, at least in part, to the 
perceived ineffectiveness of school 
districts. States and the federal 
government, more and more, provide 
funding directly to schools, rather than 
via district central offices. And in 1994, 
the Annenberg Foundation's $500 
million "Challenge" pledge, the largest 
gift to urban public education ever 
made, bypassed the school district's 
central offices as a mechanism of 
reform in nearly every site where it 
operated. 

The School District of Philadelphia was 
the one exception in the latter case. 
Philadelphia's Children Achieving, 
funded through a $150 million 
Annenberg Challenge grant, 1 was a 
sweeping "systemic reform" initiative. 2 



1 Of the $150 million, $50 million was provided by 
the Annenberg Foundation, and $100 million from 
local matching contributions. 

2 In this report, the term "systemic reform" is used 

to connote an array of reform policies, including 



A relatively new theory of education 
reform, systemic reform eschews a 
school-by-school reform approach and 
relies on the use of content and 
performance standards, 
decentralization, and accountability 
mechanisms to transform entire school 
systems. Led by central office 
personnel, Children Achieving was 
among the first attempts to put a 
systemic reform theory to a test. 

In this report, we examine the role of 
the central office in this reform and 
describe its evolution over the course 
of Children Achieving. 3 The first section 
of the report recounts how conflicts 
arose about the theory of systemic 
reform, some of the Philadelphia 
reformers' underlying beliefs and 
values, and the new roles envisioned for 
the central office in the Children 
Achieving reform plan. Central office 
staff struggled with the competing 
demands of accountability, 
decentralization, and equity. What was 
envisioned as a coherent and 
interconnected set of strategies was 
not always implemented or perceived in 
that way. One result of the confusion 
and inconsistencies in reform theory, 
design, and implementation was a 
gradual, but consistent, retreat from 



content and performance standards, accountability 
mechanisms, and decentralization. We use the term 
"system-wide reform" to refer to reform across 
entire districts, which is not necessarily "systemic 
reform" (i.e., based on standards, accountability, and 
decentralization, as defined above). We use the term 
"standards-based" when we are discussing the 
implications of content and performance standards in 
particular. 

3 Results of the reforms are reported in companion 
documents that will be published in mid-2001. For 
the most up-to-date information on these 
publications, please contact CPRE at (215) 573-0700 
or visit our web site: www.gse.upenn.edu/cpre/. 
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what initially was a key facet of the 
plan: decentralization of authority from 
the central office to schools. 

The second part of the report examines 
the capacity of the school district to 
effectively support the reforms and 
discusses the contextual issues that 
affected implementation. We argue 
that insufficient attention to the 
knowledge, skills, and beliefs of 
educators at all levels of the system as 
well as the alienation of important 
district partners, limited the human, 
social, and fiscal capital available to the 
district. This left the district without the 
resources it needed to take on a reform 
of the magnitude of Children 
Achieving. 

The Philadelphia story is neither one of 
unbridled success nor irredeemable 
failure. In addition to notable test score 
improvements highlighted in other 
CPRE reports, Philadelphia created 
content standards in every subject, 
benchmarks at important transition 
grades, and curriculum frameworks for 
every grade. The District also designed 
and implemented complex 
accountability measures for schools, 
reorganized into clusters of schools and 
small learning communities, provided 
new learning opportunities for teachers, 
and wrestled with the implementation 
of systemic reform. But the hopes that 
greeted this ambitious reform when it 
began were never fulfilled. After David 
Hornbeck resigned as Superintendent 
in August 2000, Children Achieving was 
all but dismantled. Our analysis 
suggests that a combination of factors 
contributed to the demise of Children 
Achieving including: flaws in the theory 
of action, flaws in implementation, lack 



of capacity and lack of attention to 
building capacity, and inconsistency by 
stakeholders about the beliefs and 
values underlying the reform effort. The 
final section of the report draws on the 
experience of Philadelphia, both the 
successes and the challenges, to 
provide lessons for other districts taking 
on systemic reform. 

BACKGROUND TO 
PHILADELPHIA'S CHILDREN 
ACHIEVING REFORM 

The ultimate goal of the Children 
Achieving initiative was to do what "no 
city with any significant number and 
diversity of students" had ever done; 
help "a large proportion of its young 
people achieve at high levels." 4 To do 
that, David Hornbeck, the 
Superintendent who was the primary 
architect of the reform effort, sought to 
transform the entire District, from 
schools to central office. Supported by 
the Annenberg Challenge grant, he 
argued that the District could raise 
student achievement and improve 
teaching and learning by implementing 
standards and a strong accountability 
system, empowering schools by moving 
authority for instructional decisions 
away from the central office, and 
building capacity by providing a host of 
supports for teachers and students. 
Attaining the goal of improved student 
achievement meant implementing 
throughout the District — all at one 
time — a complex set of reform 
initiatives designed to work in concert. 



4 

School District of Philadelphia, Children Achieving 
strategic action design. Philadelphia: Author, 1995, 
P- 
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In our first report on the initiative, we 
described Children Achieving's theory 
of action this way: 

Given high academic standards and 
strong incentives to focus their efforts 
and resources; more control over 
school resource allocations, 
organization, policies, and programs; 
adequate funding and resources; more 
hands-on leadership and high-quality 
support; better coordination of 
resources and programs; schools 
restructured to support good teaching 
and encourage improvement of 
practice; rich professional development 
of their own choosing; and increased 
public understanding and support; the 
teachers and administrators of the 
Philadelphia schools will develop, adopt 
or adapt instructional technologies and 
patterns of behavior that will help all 
children reach the District's high 
standards . 5 

Superintendent Hornbeck accepted this 
description as an accurate statement of 
his theory of action . 6 

The statement above also reflects, in 
essence, the prevailing theory of 
systemic reform . 7 "Systemic reformers" 
argue that previous attempts at reform 

5 Consortium for Policy Research in Education, 
Research for Action, and OMG Center for 
Collaborative Learning, A first-year evaluation report 
of Children Achieving : Philadelphia's education 
reform. Philadelphia: Greater Philadelphia First, 

1996. 

6 Field notes, September 1996. 

7 J. Christman, T. Corcoran, E. Foley, and T. Luhm, 
"Philadelphia's Children Achieving initiative: The 
promise and challenge of systemic reform in an 
urban school district." In W. Boyd (ed.), Reforming 
urban school governance: Responses to the crisis of 

performance. Westport, CT: Ablex/Greenwood 
Publishing, in press. 



have largely failed because they were 
too incremental, too narrowly framed, 
and did not attempt to alter the 
"system" itself. They assert that 
previous reforms did not institutionalize 
high expectations for students and 
teachers and did not fundamentally 
alter the roles of those in charge of 
governance and management 
structures. They conclude that a more 
comprehensive strategy that sets 
academic standards, monitors 
accountability, and coheres policy is 
necessary. 

BELIEFS AND VALUES 
UNDERLYING THE REFORM 

Philadelphia was among the first urban 
school districts to take a systemic 
approach to school reform and to test 
this new theory of school improvement. 
In addition to systemic theory, outlined 
in a ten-point plan , 8 Children Achieving 
rested on a particular set of beliefs and 
values. We found these values 
articulated in District documents, in 
speeches made by Superintendent 
Hornbeck and other leaders, in 
interviews with School District staff, and 
in discussions at policy meetings. They 
included: 

Primacy of results. Results are what 
matter; how they are achieved is less 
important. 

Equity is paramount. The School 
District must be an advocate for the 
poor children it serves. Equity — of 
academic expectations, learning 



8 For a detailed description of the Children Achieving 
reform plan, please see our previous reports or 
School District of Philadelphia, Children Achieving 
strategic action design. 
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opportunities and achievement 
outcomes — is a paramount objective. 
All children can learn, and "all" means 
"all." 

School personnel need autonomy to 
meet the needs of their students. 
Those working closest to students know 
what's best for students, and want and 
need the freedom and authority to act 
on their decisions. Hence, central 
authorities should not prescribe the 
means to achieve the goals lest they 
inhibit local innovation. 

Strong incentives are necessary. To 
spur action at the "cluster " 9 and school 
level, strong incentives must be 
developed. Incentives might include 
rewards and sanctions for performance 
as well as rewards and sanctions for 
adopting particular strategies or 
behaviors. 

Do it all at once. Reform in all aspects 
of the system must occur 
simultaneously and immediately to 
achieve significant results. 

These beliefs and values were not 
equally important nor consistently 
apparent over the course of the reform. 
Depending on the central office figure 
leading the effort, the year, or the goal 
of the initiative, one core belief might 
be emphasized over another. 
Sometimes seemingly conflicting beliefs 
co-existed. We will return to these 
beliefs and values throughout this 



9 Clusters were created under Ch/7dren Achieving 
and are the District's intermediary organizational unit 
between the central office and the schools. Under 
Ch/7dren Achieving there were 22 clusters in 
Philadelphia, each organized around a 
comprehensive high school and the elementary and 
middle schools that feed into it. 



report, demonstrating how they shaped 
central office policy and reform 
implementation, the relationship 
between schools, clusters, and the 
central office, and the roles of central 
office leaders. 



